Pitt@TREC08: An Initial Study of Collaborative Information Behavior in E-Discovery

The University of Pittsburgh team participated in the interactive task of Legal Track in TREC 2008. We designed an experiment to investigate into the collaborative information behavior (CIB) of the group of people working on e-discovery task provided by Legal Track in TREC 2008. Through the study, we identified three major characteristics of CIB in e-discovery. 1) Frequent communication among participants 2) division of labor is common; and 3) “awareness” is important among participants. Based on these insights, we also propose a set of essential technologies and functions for retrieval systems that support CIB. 1.1 Collaborative Information Behavior It is well recognized that people act in a social and organizational context when trying to solve information seeking problems (Karamuftuoglu, 1998; Munro, Hook, & Benyon 1999; Soininen & Suikola, 2000). However, in most collaboration environments, information behavior is still commonly perceived as operating at the individual level (Sonnenwald & Pierce, 2000). Therefore, in order to understand aspects of collaborative activities from an information seeking and retrieval perspective, we need to investigate the manifestations of collaboration (Hansen & Jarvenlin, 2005). Although no definition of collaborative information behavior (CIB) has been universally accepted, for the sake of our discussion, we adopt a definition of CIB as “activities that a group or team of people undertake to identify and resolve a shared information need” (Poltrock, Dumais, Fidel, Bruce, & Pejtersen, 2003). Two important aspects of CIB are revealed in this definition. The first is collaboration where people working together to seek information. The second aspect is about resolving one common information need, which includes seeking, retrieving, and using information to solve a problem (Reddy & Jansen 2008). This definition helps us to establish that CIB is as common and natural as individual information behavior (IIB) (Hansen & Jarvenlin, 2005; McKenzie, 2003; Talja, 2002) CIB studies have been conducted in many different settings, including academia, industry, medicine, military and everyday life. Talja (2002) conducted a study through empirical observation and showed that there were different types of information sharing in academic communities. A study of two teams engaged in the design of computer-related products focused on how team members collectively sought and shared external information acquired within the team (Poltrock, Grudin, Dumais, Fidel, * Corresponding author: email: [email protected] Bruce, & Pejtersen, 2003). In a study of information behavior in military command and control teams, Sonnenwald and Pirerce (2000) studied collaboration in dynamic situations with rapidly changing information and a need for continuous information exchange. Through a study of CIB of two healthcare teams, Reddy and Jansen (2008) proposed a model for understanding CIB in context. Within everyday life information seeking (ELIS) studies, McKenzie (2003) found that people routinely assist each other in solving information problems. To the best of our knowledge, there are few studies of CIB that have been conducted in a legal search or related setting. Therefore, our study presented in this paper, which was conducted under the e-discovery task in Legal track of TREC 2008, will be an important contribution to the CIB literature in legal domain. 1.2 E­Discovery and TREC Legal Track Discovery of the relevant information gathered about a topic in dispute is at the core of the litigation process (Hickman, 1947). With the help of modern information technology and the common practices of using digital documents, the discovery process need to identify and produce relevant information has evolved from the manual review of paper documents to the one involving vast volumes of electronic documents (Baron, 2007). It is under this circumstance that TREC Legal track started in 2006. The legal track has two tasks. The ad hoc task is designed to facilitate comparison of retrieval systems, whereas the other task – the interactive challenge task – was to model more realistically the way in which retrieval queries might be generated, refined, and applied in the e-discovery domain [TREC-2008 legal track guidelines]. In real e-discovery processes, there is a lead attorney who is in charge of overseeing a large document-review effort and for vouching for the completeness and accuracy of the produced collection. The attorney often hires an e-discovery firm or team to gather all the relevant documents from the full document collection implicated by the matter. The design of interactive challenge task resembles this situation, and the research team involving in the task acts as the hired e-discovery firm, and the track organizers act as the lead attorney to provide authority on the issues related to the relevance and the scope of the discovery process. Therefore, e-discovery tasks in general, and the interactive challenge task in specific, are challenging and complex information seeking tasks that requires a group of people to collaborate. The goal of our participation in this year’s TREC Legal Track, therefore, is to investigate CIB in e-discovery, and generate a set of insights into CIB, and a set of recommendations for tools that truly support CIB in e-discovery and in information seeking in general. 2 Methodology and Experimental Setting 2.1 Tasks and Experiment Procedure Our investigation of CIB activities in e-discovery involves a group of people working on the e-discovery tasks provided by Legal Track. The objective of each task is to find all the relevant documents from a Tobacco document collection. The Legacy Tobacco Documents Library (LTDL) contains more than 9.7 million documents (50+ million pages) created by major tobacco companies related to their advertising, manufacturing, marketing, sales, and scientific research activities. The participants used the search tool provided by LTDL to search for relevant documents. We designed our study to have two phases. In the first phase, two participants worked on the following e-discovery topic from the TREC-2008 Legal Track Interactive task: Documents referring to marketing or advertising restrictions proposed for inclusion in, or actually included in, the Master Settlement Agreement ("MSA"), including, but not limited to, restrictions on advertising on billboards, stadiums, arenas, shopping malls, buses, taxis, or any other outdoor advertising. In this first phase, the participants worked separately in different places, and there was no time limitation of finishing the task. Participants could do whatever they want to complete the task. It was in the second phase, the same participants worked collaboratively on a second e-discovery task from the TREC-2008 Legal Track Interactive task which follows: All documents which describe, refer to, report on, or mention any "in-store," "on-counter," "point of sale," or other retail marketing campaigns for cigarettes. The study in this phase was structured to observe and document the types of interactions between the participants when working as a team. Specifically we were interested in the flow of information between the two participants which contribute to accomplishing the team’s task. The two participants were working at their own desks which are approximately two feet apart, facing each other. In order for one participant to see the other’s computer screen, the screen had to be turned around (easily done) or the person had to walk around to that side of the line of desks and look at it. No one else was working in the office when the study was taking place, so verbal communications also played a large role in the information exchange process. Our experiments actually focused on the second phase which involved CIB. However, phase 1 was important because it allowed participants getting familiar with the e-discovery tasks and the collection. It also helped the participants make comparison to the experience between IIB and CIB.